Satisficing in Time-Sensitive Bandit Learning

Time-Sensitive Bandit Learning and Satisficing Thompson Sampling

The literature on bandit learning and regret analysis has focused on contexts where the goalis to converge on an optimal action in a manner that limits exploration costs. One shortcomingimposed by this orientation is that it does not treat time preference in a coherent manner.Time preference plays an important role when the optimal action is costly to learn relative tonear-o...

Satisficing: A ‘Pretty Good’ Heuristic

One of the best known ideas in the study of bounded rationality is Simon’s satisficing; yet we still lack a standard formalization of the heuristic and its implications. We propose a mathematical model of satisficing which explicitly represents agents’ aspirations and which explores both singleperson and multi-player contexts. The model shows that satisficing has a signature performanceprofile ...

Cost-sensitive decision tree learning using a multi-armed bandit framework

The Blinded Bandit: Learning with Adaptive Feedback

We study an online learning setting where the player is temporarily deprived of feedback each time it switches to a different action. Such model of adaptive feedback naturally occurs in scenarios where the environment reacts to the player’s actions and requires some time to recover and stabilize after the algorithm switches actions. This motivates a variant of the multi-armed bandit problem, wh...

Large-Scale Bandit Problems and KWIK Learning

We show that parametric multi-armed bandit (MAB) problems with large state and action spaces can be algorithmically reduced to the supervised learning model known as “Knows What It Knows” or KWIK learning. We give matching impossibility results showing that the KWIKlearnability requirement cannot be replaced by weaker supervised learning assumptions. We provide such results in both the standard...